Data Analytics
Data Analytics - Usage, Types
Data Analytics
Data Analytics is the process of collecting, cleaning, organizing, analyzing, and interpreting data to discover useful information, identify patterns, support decision-making, and solve real-world problems. In the real world several data are available. Data like customers' click stream data, stock data, transaction history etc. Those data can be analyzed to find different kinds of information which can be of great value in decision making
Data Analytics is the science of converting raw data into meaningful information for better decision-making.
Data Analytics Process
- Data collection - Gather data from databases, websites, surveys, sensors, etc.
- Data cleaning - remove errors, duplicates, and missing values. Real world may be incomplete, that is some records may have no values for some attributes. Data may be confusing like age of customer is written as 120 years.
- Data transformation - convert data into a suitable format. Data mining and Data analytics algorithms expect data in specific format or range, so data given in one format have to be converted to another format. For example, salaries given in thousands need to be converted to range 0 to 1.
- Data analysis - apply statistical and machine learning techniques
- Interpretation and visualization - Present findings using charts, dashboards, and reports.
- Decision making - use insights to improve business or organizational performance
Usage of Data analytics
Data analytics is used in almost every sector like business, banking, healthcare, education, government, agriculture, manufacturing, social media, transportation, E-commerce.
Importance of Data analytics
- Supports better decision-making. Information about customer behavior can be used to develop marketing strategy.
- Improves business performance. Informaton can be used to ease daily occuring tasks and activities.
- Reduces operational costs
- Understands customer behavior
- Predicts future trends. Based on past performance future trend can be predicted.
- Detects fraud and risks
- Provides competitive advantage
The most common data mining and analytics tasks include:
- Association analysis
- Classification
- Clustering
- Outlier analysis
Association analysis
Association Analysis is a technique used to discover relationships or associations among items that frequently occur together in a dataset.
By analyzing transaction database, association between products can be found. Association like: in lots of transactions (around 15% of total transactions) where TV is purchased, additional remote control is also purchased.
Apriori, Frequent Pattern (FP) Growth Tree are algorithms for association analysis.
Applications
- Product recommendation. Additional remote control can be recommended to customer who purchase TV.
- Cross-selling / Up selling. A customer searching for 32 inch TV can be shown 40 inch TV if that customer's purchasing behavior is known.
- Online-shopping suggestions
- Website navigation analysis
- Medical diagnosis
- Library book, movie recommendation
Advantages
- Finds hidden relationships
- Improves sales
- Helps product placement
- Supports recommendation systems
Limitations
- Generates many rules
- Some rules may be meaningless
- Requires large datasets
classification
Classification is a supervised machine learning technique that assigns data into predefined categories or classes. Classes can be buyers, non-buyers and many more. Decision Tree, Neural Network, Bayesian Classifiers are examples of classification algorithms
Classification algorithm first creates model using training dataset. Then test dataset is used to test the model. If the result of test is convincing, then the model is used to predict class of new data for which class is unknown.
After learning, the model predicts the class of new emails.
Applications
- Disease diagnosis
- Loan approval
- Fraud detection
- Email filtering
- Face recognition
- Sentiment analysis
Advantages
- Easy prediction
- High accuracy
- Suitable for labeled data
Limitations
- Requires labeled datasets
- Performance depends on training data quality
Clustering
Clustering is an unsupervised learning technique that groups similar objects together without predefined labels. Clusters are groups. Objects within the same cluster are more similar to each other than to those in other clusters. Algorithms like Partitioning Around mean (PAM), hierarchical algorithms, Density-based algorithms can be used for clustering.
A shopping website groups customers based on purchasing behavior. Based on the customers purchasing behavior customers several clusters like: big spenders, budget spenders, medium spenders can be created. Different marketing strategies can be developed for each cluster to improve sales.
Applications
- Customer segmentation
- Image segmentation
- Social network analysis
- Medical research
- Market research
- Document clustering
Limitations
- Choosing the correct number of clusters can be difficult. That is how many clusters (3 / 4 or many clusters) are required.
- Results depend on the algorithm and parameter settings
Outlier Analysis
Outlier Analysis identifies data objects that differ significantly from the majority of the data. Outliers do not belong to any cluster. An outlier is an observation that is unusually different from other observations
Why outliers matter?
Outliers may indicate:
- Fraud
- Data entry errors
- Equipment failure
- Rare events
- Exceptional performance
Applications
Advantages
- Detects abnormal behavior
- Improves data quality. By removing outliers data qaulity can be improved.
- Helps identify fraud and anomalies
Limitations
- Some outliers are valid observations rather than errors
- Different methods may identify different outliers